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Executive  Summary 


Our  work  in  RT-155  has  continued  tool  development  of  the  System  of  Systems  Analytic 
Workbench  (SoS-AWB)  and  transitional  activities  with  key  collaborators.  The  tools  include: 
System  Operational  Dependency  Analysis/System  Developmental  Dependency  Analysis 
(SODA/SDDA),  Multi-Stakeholder  Dynamic  Optimization  (MUSTDO),  System  Importance 
Measures  (SIMs),  and  Robust  Portfolio  Optimization  (RPO). 

The  workbench  provides  a  means  of  addressing  SoS  practitioners'  'archetypal  questions'  in 
scenarios  where  size  and/or  interdependencies  between  systems  are  high.  The  workbench  also 
enables  analysis  of  capability  and  risk  during  the  conduct  of  SoS  evolution.  Our  common  use  case 
demonstration  problem  involves  a  Naval  Warfare  Scenario  (NWS)  that  is  based  on  the 
architecture  of  the  Littoral  Combat  Ship  (LCS).  Each  of  the  tools  has  used  this  common  case  study 
to  illustrate  the  value  in  each  tool  performing  various  types  of  SoS  level  architectural  analysis,  as 
evidenced  by  prior  literature  and  extensive  documentation  for  predecessor  RT-36,  RT-44b,  RT- 
108,  and  RT-134  tasks. 
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Summary  Of  RT-155  Achievements 


While  continuing  research  on  each  of  the  methods,  our  FY16  effort  emphasized  transitioning 
of  the  AWB  for  test  in  practical  settings  with  our  key  collaborators,  and  dissemination  to  the 
broader  community.  Our  efforts  have  resulted  in  the  following  areas  of  coordination  and 
dissemination  (details  of  these  have  been  documented  in  RT-155  bi-monthly  reports): 


Output 

Description  of  Dissemination 

Conference 

Publications 

CSER2017 

"A  robust  portfolio  optimization  approach  using  parametric  piecewise  linear 
models  of  system  dependencies"  15*  Annual  Conference  on  Systems 
Engineering  Research  [CSER],  Redondo  Beach,  CA,  March  23-25,  2017 
[accepted] 

lAC  2016 

"Understanding  Human  Space  Exploration”  67*  lAF  International 
Astronautical  Congress  Guadalajara,  Mexico,  26-30  Sept  2016  - 

[recommended  by  conference  committee  for  publication  to  Acta  Astronautica] 

Journal 

Publications 

IEEE  Systems 

"Multi-Stakeholder  Dynamic  Optimization  for  Acknowledged  System-of- 
Systems  Architecting"  [submitted] 

Research  in  Engineering  Design 

Guariniello,  C.,  DeLaurentis,  D.,  "Supporting  design  via  the  System  Operational 
Dependency  Analysis  methodology",  Research  in  Engineering  Design  Vol.28 
(2017),  pp.  53-69 

Webinar 

/Panels 

Systems  Engineering 

Resilient  System-of-System  Design  using  System  Importance  Measures,  [Will 
prepare  for  submission] 

SoSCIE  webinar,  Center  for  Education  and  Research  in  Information  Assurance 
and  Security  [CERIAS] 
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Extension  of  SoS-AWB  Methodologies 


We  have  extended  each  tool  in  the  SoS-AWB  and  refined  them  through  our  exchanges  with 
collaborative  partners.  Primary  changes  and  value  added  in  the  methods  of  the  SoS-AWB 
involves  the  following: 


1.  Combined  RPO-SODA  method  -  We  have  integrated  the  network  interdependency 
modeling  capabilities  of  SODA  within  the  portfolio  optimization  construct  of  the 
RPO— this  involved  mathematical  modeling  development  of  necessary  equations  that 
can  explicit  introduce  SODA'S  piece-wise  linear  construct  within  RPO.  Theoretical 
details  and  an  example  application  are  included  in  our  publication  titled  "A  robust 
portfolio  optimization  approach  using  parametric  piecewise  linear  models  of  system 
dependencies"  which  we  will  present  at  the  CSER  2017  conference.  We  have  included 
a  demonstrative  version  of  the  joint  framework  for  execution  through  the  RPO  GUI 
interface. 


2.  MUSTDO  -  Extended  and  validated  the  mathematical  formulation  of  optimal  transfer 
contract  scheme  being  used  within  MUSTDO;  Explored  practical  interpretations  of  the 
transfer  contract  concept. 

3.  SIMs  -Validated  existing  framework  with  an  improved  ABM  simulation  and  improved 
existing  resilience  map  to  better  reflect  critical  systems  and  potentially  effective 
mitigations. 

4.  SODA/SDDA  -  Added  ability  to  model  cyclic  networks  (SODA).  Added  ability  to  include 
external  factors  (SDDA) 
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Engagement/Interaction  Summary 


SERC-  We  have  continued  our  collaborative  exchanges  with  various  members  of  SERC 
community  as  and  when  needed  towards  development  and/or  dissemination  of  the  SoS- 
AWB. 


MITRE-  We  have  had  detailed  exchanges  with  personnel  at  MITRE  on  use  of  the  SoS-AWB 
toolset  for  internal  cases  run  locally  at  MITRE.  The  interactions  were  with  MITRE  members: 
(Laura  AntuI,  Ryan  Jacobs,  Matt-Cotter  and  Aleksandra  Markina-Khusid)  and  leads:  (Dr. 
Judith  Dahmann  and  Tom  Wheeler).  Interactions  involved  primary  adaptation  of  SODA  and 
RPO  toolset  for  application  on  internal  case  problems  to  MITRE  environment. 


Johns  Hopkins  University  Applied  Physics  Laboratory  (JHUAPL)  -  Transition  of  SERC  Analytic 
Workbench  Technology  into  Government  Service  by  JHUAPL.  The  application  effort  involves 
modeling  an  existing  messaging  network  to  perform  real-time  situational  awareness.  The 
Analytical  Workbench's  combination  of  dependency  metrics  (strength,  criticality,  and  impact)  will 
be  leveraged  within  their  own  model  of  system  effectiveness  to  measure  network  performance 
in  real-time.  These  models  will  then  be  linked  to  a  visualization/dashboard  system  that  will  allow 
operators  to  quickly  and  graphically  determine  consequences  to  related  components  based  on 
the  dependencies.  We  have  attached  the  JHUAPL-SERC-Article.pdf  that  covers  the  scope  of 
technology  transfer  for  the  SoS-AWB. 


Naval  Surface  Warfare  Center  Dahlgren  Division  (NSWCDD)- 

With  our  CRADA  finalized  with  Navy  NSWCDD,  we  made  substantial  progress  (with  Dahlgren 
POC  Mary  Ann  Cummings),  including  the  following: 


•  Detailed  technical  exchanges  and  demos  to  establish  understanding  and  requirements  for 
use  of  the  SoS  AWB  at  NSWCDD. 

•  Developed  SoS-AWB  in  accordance  with  exchanges,  resulting  in  software  being  shared  with 
NSWCDD  (POC:  Adam  Bahrainwala).  More  specifically,  Adam  Bahrainwala  demonstrated  use 
of  internal  simulation  based  data  being  used  in  tandem  with  SODA  toolset  for 
interdependency  analysis. 


NASA  Marshall  Space  Flight  Center  -  Cooperative  agreement  in  place  to  use  methods  in  SoS-AWB 
(RPO,  SODA,  SDDA)  to  analyze  space  exploration  architectures. 
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Software  Dissemination  and  NanoHub  Deployment 


We  have  made  prototype  version  of  our  tools  available  through  the  nanohub.org  portal,  hosted 
at  Purdue  University.  The  team  worked  on  the  development  and  compilation  of  tools  for  the 
hub  environment,  including  necessary  edits  to  relevant  toolsets  and  GUI  interfaces,  to  facilitate 
compatibility  in  this  hosted  environment.  We  have  60  users  thus  far,  with  a  total  of  ~700+  runs 
of  the  SoS  AWB.  (Please  see  attached  file:  nanohub-usage.pdf  for  global  usage  report  and 
simulation  runs  statistics  of  our  toolset). 


The  tool  is  open  to  the  public  and  users  can  sign  up  for  a  free  account  at  nanohub.org  to  access 
the  System  of  Systems  Analytic  Workbench  set  of  tools.  Additionally,  we  are  delivering  an 
updated  copy  of  our  software  to  the  DoD  SERC,  in  accordance  with  the  terms  of  our  statement 
of  work. 


In  summary,  we  have  made  progress  in  refining  methods/fixing  bugs  through  interactions  with 
collaborator  entities  using  demonstrative  versions  of  our  SoS  AWB.  Further,  we  see  continued 
use  of  the  online  version  of  our  toolset  as  evidenced  by  the  published  statistics.  Our  transitional 
and  collaborative  activities  with  MITRE  Corporation,  Naval  Surface  Warfare  Center  Dahlgren 
Division  (NSWCDD)  and  Johns  Hopkins  University  Applied  Physics  Laboratory  (JHUAPL)  have 
been  successful  in  that  the  respective  collaborators  have  demonstrated  use  of  the  Purdue  SoS- 
AWB  in  native  environments.  As  of  this  report,  we  are  in  the  middle  of  executing  an  official 
technology  transfer  of  IP  (limited)  to  JHUAPL,  in  accordance  with  relevant  RT-155  contract 
agreement  clauses. 


Guide  to  the  Various  Products  attached  to  the  Summary  Final  Report 

•  Papers 

o  SODA_RIED.pdf  -  Supporting  design  via  the  System  Operational  Dependency 
Analysis  methodology  -  Journal  paper,  published  in  Research  in  Engineering 
Design. 

o  MUSTDO_Methodogy_UnderReview.pdf  -  Draft  paper,  submitted  for 
consideration  to  IEEE  Systems  journal 

o  IAC-16-A3.19.pdf-  Paper  published  at  lAC  2016  conference  on  SODA  methodology 
and  application 


•  Code  documentation 

o  SODA_GUI_Manuol.docx:  description  of  the  Graphic  User  Interface  for  SODA 
analysis,  including  possible  inputs,  choice  of  methods,  resulting  plots,  output  files. 
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o  SDDA_GUI_Manual.docx:  description  of  the  Graphic  User  Interface  for  SDDA 
analysis,  including  possible  inputs,  choice  of  methods,  resulting  plots,  output  files, 
o  RPO_Manual_New_Version.docx:  description  of  Graphic  User  Interface  for  Robust 
Portfolio  Optimization  toolset,  including  required  inputs,  choice  of  methods, 
output  numbers  and  plots  and  other  pertinent  information, 
o  MUSTDO_GUI_Manual.docx:  description  of  the  Graphic  User  Interface  for 
MUSTDO  framework,  including  possible  inputs  and  resulting  plots, 
o  SIM_GUI_Manual.docx:  description  of  the  Graphical  User  Interface  for  SIM 
framework,  including  required  inputs,  analysis  methods,  type  of  outputs  and  other 
definitions. 

•  Technical  Summaries  of  Methods: 

o  A  technical  summary  of  methods,  processes  and  tools  in  the  SoS-AWB  is  given 
in  the  SERC  Sponsor  Research  Review  RT-155  slides  as  attached  to  this  report. 
(Purdue-RT-SSRR-2016-Final.pdf) 


Future  Plans 


Our  work  in  this  RT-155  effort  has  presented  value  added  to  our  key  collaborators  and  the 
broader  community.  We  have  engaged  SERC  on  a  path  forward  for  further  transitional  activities 
and  are,  at  the  time  of  this  report  being  generated,  proposing  follow-on  transitional  activity  and 
development  work  under  the  auspices  of  a  new  RT-178  effort. 
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